{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dogs of the Dow Theory 「狗股」\n",
    "\n",
    "\n",
    "> 狗股是投資美國股市的一個選股策略。原始狗股在道瓊斯工業指數內的股票里選取高股息率股票組合，現有其他變種，如小狗股等。也有人用類似的選股方式用在港股。\n",
    ">\n",
    ">　　對一心“執平貨”的投資者來說，股息率（dividend yields）是一個常用的指標；簡而言之，股息率愈高愈“抵買”，反之亦然。根據這個理念發展出來的“道指狗股理論”（Dogs of the Dow Theory），為一些懶於（或不懂）分析公司基本因素、從中尋找“超值”股票，卻又希望跑贏大市的散戶，提供了一個方便實用的選擇。\n",
    ">\n",
    ">　　狗股理論是美國基金經理邁克爾·奧希金斯（Michael O'Higgins）於1991年提出的跑贏大市投資策略。具體的做法是，投資者每年年底從道瓊斯工業平均指數成份股中找出10只股息率最高的股票，新年伊始買入，一年後再找出10只股息率最高的成分股，賣出手中不在名單中的股票，買入新上榜單的股票，每年年初年底都重覆這一投資動作，便可獲取超過大盤的回報。\n",
    ">\n",
    ">　　據有關統計，1975至1999年運用\"狗股理論\"，投資的平均複利回報達18%，遠高於市場3%的平均水平。但不幸的是，2008年，在百年一遇的金融危機中，10只“狗股”平均跌幅超過41％，跑輸了道指（34％）。不過，就像百年老店雷曼兄弟倒了一樣，2008年利用“狗股理論”的投資者輸得這麼慘，應該也屬於小概率事件。\n",
    ">\n",
    ">From [MBA 智库百科](https://wiki.mbalib.com/zh-tw/%E7%8B%97%E8%82%A1%E7%90%86%E8%AE%BA)\n",
    "\n",
    "这里，我们通过两个量化交易库来帮助我们进行选股和回测 tushare & abupy, 当然，以后可能会用到 JQData"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "NumExpr defaulting to 8 threads.\n"
     ]
    }
   ],
   "source": [
    "# 基础库导入\n",
    "\n",
    "from __future__ import print_function\n",
    "from __future__ import division\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "warnings.simplefilter('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import tushare as ts\n",
    "import datetime\n",
    "\n",
    "import os\n",
    "import sys\n",
    "# 使用insert 0即只使用github，避免交叉使用了pip安装的abupy，导致的版本不一致问题\n",
    "sys.path.insert(0, os.path.abspath('../'))\n",
    "import abupy\n",
    "\n",
    "# 使用沙盒数据，目的是和书中一样的数据环境\n",
    "# abupy.env.enable_example_env_ipython()\n",
    "\n",
    "from abupy import AbuFactorSellNDay, AbuFactorBuyWD, AbuPickStockNTop\n",
    "from abupy import AbuFactorBuyBreak, AbuFactorAtrNStop, AbuFactorPreAtrNStop, AbuWeekMonthBuy\n",
    "from abupy import abu, AbuFactorCloseAtrNStop, ABuProgress, AbuMetricsBase, EMarketTargetType\n",
    "from abupy import AbuSymbolCN, ABuSymbolPd\n",
    "from abupy import EMarketDataFetchMode, EMarketSourceType, EDataCacheType"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Getting data:]############################################################Basic Selection: ['000825', '000709', '601000', '000040', '601005', '600308', '601169', '600859', '600269', '000683', '601186', '000039', '000415', '601077', '000778', '600694', '002133', '600716', '601390', '600188', '600348', '600012', '600282', '002092', '601398', '601009', '601838', '600926', '600000', '601288', '600623', '601229', '600051', '600382', '600782', '601166', '000926', '601998', '600153', '600350', '601939', '600638', '601669', '000959', '600997', '600823', '000898', '603518', '600016', '000980', '002936', '300089', '600971', '600019', '600708', '601818', '601997', '601988', '600808', '600919', '600067', '603323', '601577', '601328', '600743', '600376', '002619', '601699', '000791', '600657', '000069', '601800', '600123', '600569', '600015', '000402', '600740', '601828', '600231', '600665', '000900', '601588', '000732', '000620', '601101'] Totally: 85\n"
     ]
    }
   ],
   "source": [
    "## Set up the date\n",
    "today = '20' + datetime.date.today().strftime('%y%m%d')\n",
    "\n",
    "index_all = ts.get_today_all()\n",
    "\n",
    "df1 = index_all.copy()  #建立一个备份\n",
    "n = 200  #选择前n个数据\n",
    "\n",
    "df1['a']=[('ST' in x )for x in df1.name.astype(str)]  #先给ST股票做标记a\n",
    "df1=df1.set_index('a')  #将a设置为索引 \n",
    "df1=df1.drop(index=[True]) #删除ST股票\n",
    "df1=df1.reset_index(drop=True) #重建默认索引\n",
    "\n",
    "#删除业绩亏损的股票\n",
    "df1 = df1[df1.per >0]\n",
    "\n",
    "#删除净资产为负的股票\n",
    "df1 = df1[df1.pb >0]\n",
    "\n",
    "#选取市盈率前 200 名股票\n",
    "df2 = df1.sort_values(by=['per'],ascending=True).head(n)\n",
    "#print(len(df2))\n",
    "#选取市净率 200 名股票\n",
    "df3 = df1.sort_values(by=['pb'],ascending=True).head(n)\n",
    "#print(len(df3))\n",
    "#生成股票代码集合，进行集合运算\n",
    "g2 = set(df2.code) #低市盈率股票代码\n",
    "#print(len(g2))\n",
    "g3 = set(df3.code) #低市净率股票代码\n",
    "#print(len(g3))\n",
    "g = g2&g3 #集合交运算\n",
    "#print(len(g))\n",
    "pending_list = list(g)  #把集合转为列表\n",
    "print('Basic Selection:',pending_list, \"Totally:\", len(pending_list))\n",
    "\n",
    "pending_list = pd.DataFrame({\"Code\":pending_list})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>code</th>\n",
       "      <th>name</th>\n",
       "      <th>changepercent</th>\n",
       "      <th>trade</th>\n",
       "      <th>open</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>settlement</th>\n",
       "      <th>volume</th>\n",
       "      <th>turnoverratio</th>\n",
       "      <th>amount</th>\n",
       "      <th>per</th>\n",
       "      <th>pb</th>\n",
       "      <th>mktcap</th>\n",
       "      <th>nmc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>688399</td>\n",
       "      <td>硕世生物</td>\n",
       "      <td>5.310</td>\n",
       "      <td>77.35</td>\n",
       "      <td>75.00</td>\n",
       "      <td>79.75</td>\n",
       "      <td>75.0</td>\n",
       "      <td>73.45</td>\n",
       "      <td>1.5216e+06</td>\n",
       "      <td>11.4078</td>\n",
       "      <td>1.1789e+08</td>\n",
       "      <td>53.345</td>\n",
       "      <td>9.095</td>\n",
       "      <td>4.5343e+05</td>\n",
       "      <td>103173.0654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>688398</td>\n",
       "      <td>赛特新材</td>\n",
       "      <td>2.845</td>\n",
       "      <td>49.16</td>\n",
       "      <td>48.70</td>\n",
       "      <td>50.00</td>\n",
       "      <td>48.5</td>\n",
       "      <td>47.80</td>\n",
       "      <td>1.1803e+06</td>\n",
       "      <td>6.4846</td>\n",
       "      <td>5.8127e+07</td>\n",
       "      <td>68.278</td>\n",
       "      <td>9.518</td>\n",
       "      <td>3.9328e+05</td>\n",
       "      <td>89476.8042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>688396</td>\n",
       "      <td>华润微</td>\n",
       "      <td>-2.523</td>\n",
       "      <td>40.95</td>\n",
       "      <td>42.71</td>\n",
       "      <td>42.78</td>\n",
       "      <td>40.2</td>\n",
       "      <td>42.01</td>\n",
       "      <td>2.4965e+07</td>\n",
       "      <td>10.5276</td>\n",
       "      <td>1.0275e+09</td>\n",
       "      <td>83.811</td>\n",
       "      <td>62.653</td>\n",
       "      <td>4.7992e+06</td>\n",
       "      <td>971075.9659</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>688389</td>\n",
       "      <td>普门科技</td>\n",
       "      <td>1.979</td>\n",
       "      <td>19.07</td>\n",
       "      <td>19.02</td>\n",
       "      <td>19.27</td>\n",
       "      <td>18.6</td>\n",
       "      <td>18.70</td>\n",
       "      <td>2.0159e+06</td>\n",
       "      <td>5.8007</td>\n",
       "      <td>3.8315e+07</td>\n",
       "      <td>90.810</td>\n",
       "      <td>10.713</td>\n",
       "      <td>8.0514e+05</td>\n",
       "      <td>66271.6921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>688388</td>\n",
       "      <td>嘉元科技</td>\n",
       "      <td>-2.299</td>\n",
       "      <td>54.83</td>\n",
       "      <td>56.88</td>\n",
       "      <td>57.80</td>\n",
       "      <td>53.5</td>\n",
       "      <td>56.12</td>\n",
       "      <td>2.8448e+06</td>\n",
       "      <td>5.1094</td>\n",
       "      <td>1.5716e+08</td>\n",
       "      <td>53.755</td>\n",
       "      <td>5.141</td>\n",
       "      <td>1.2659e+06</td>\n",
       "      <td>305276.2124</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     code  name  changepercent  trade   open   high   low  settlement  \\\n",
       "0  688399  硕世生物          5.310  77.35  75.00  79.75  75.0       73.45   \n",
       "1  688398  赛特新材          2.845  49.16  48.70  50.00  48.5       47.80   \n",
       "2  688396   华润微         -2.523  40.95  42.71  42.78  40.2       42.01   \n",
       "3  688389  普门科技          1.979  19.07  19.02  19.27  18.6       18.70   \n",
       "4  688388  嘉元科技         -2.299  54.83  56.88  57.80  53.5       56.12   \n",
       "\n",
       "       volume  turnoverratio      amount     per      pb      mktcap  \\\n",
       "0  1.5216e+06        11.4078  1.1789e+08  53.345   9.095  4.5343e+05   \n",
       "1  1.1803e+06         6.4846  5.8127e+07  68.278   9.518  3.9328e+05   \n",
       "2  2.4965e+07        10.5276  1.0275e+09  83.811  62.653  4.7992e+06   \n",
       "3  2.0159e+06         5.8007  3.8315e+07  90.810  10.713  8.0514e+05   \n",
       "4  2.8448e+06         5.1094  1.5716e+08  53.755   5.141  1.2659e+06   \n",
       "\n",
       "           nmc  \n",
       "0  103173.0654  \n",
       "1   89476.8042  \n",
       "2  971075.9659  \n",
       "3   66271.6921  \n",
       "4  305276.2124  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用腾讯财经的数据 （比百度好）\n",
    "abupy.env.g_market_source = EMarketSourceType.E_MARKET_SOURCE_tx\n",
    "abupy.env.g_data_cache_type = EDataCacheType.E_DATA_CACHE_HDF5\n",
    "# ABuSymbolPd.make_kl_df('601988')\n",
    "\n",
    "# 设定时间区间和市场 n_jobs 就是并行数\n",
    "abu.run_kl_update(start='2019-01-01', end='2020-01-01', market=EMarketTargetType.E_MARKET_TARGET_CN, n_jobs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当然，我们应该把周期缩短一点，比如一个月一个月的筛选（毕竟政策市）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'AbuFactorBuyBreak' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-1-56449eb8ca30>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;31m# 延用周期突破策略做为买入因子\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m buy_factors = [{'xd': 21, 'class': AbuFactorBuyBreak},\n\u001b[0m\u001b[1;32m      5\u001b[0m                {'xd': 42, 'class': AbuFactorBuyBreak}]\n\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'AbuFactorBuyBreak' is not defined"
     ]
    }
   ],
   "source": [
    "cash = 10000\n",
    "\n",
    "# 延用周期突破策略做为买入因子\n",
    "buy_factors = [{'xd': 21, 'class': AbuFactorBuyBreak},\n",
    "               {'xd': 42, 'class': AbuFactorBuyBreak}]\n",
    "\n",
    "# 卖出策略也还是继续延用\n",
    "sell_factors = [\n",
    "    {'stop_loss_n': 1.0, 'stop_win_n': 3.0,\n",
    "     'class': AbuFactorAtrNStop},\n",
    "    {'class': AbuFactorPreAtrNStop, 'pre_atr_n': 1.5},\n",
    "    {'class': AbuFactorCloseAtrNStop, 'close_atr_n': 1.5}\n",
    "]\n",
    "\n",
    "def run_loo_back(choice_symbols, ps=None, n_folds=2, start=None, end=None, only_info=False):\n",
    "    \"\"\"封装一个回测函数\"\"\"\n",
    "    if choice_symbols[0].startswith('us'):\n",
    "        abupy.env.g_market_target = EMarketTargetType.E_MARKET_TARGET_US\n",
    "    else:\n",
    "        abupy.env.g_market_target = EMarketTargetType.E_MARKET_TARGET_CN\n",
    "    abu_result_tuple, _ = abu.run_loop_back(cash,\n",
    "                                           buy_factors,\n",
    "                                           sell_factors,\n",
    "                                           ps,\n",
    "                                           start=start,\n",
    "                                           end=end,\n",
    "                                           n_folds=n_folds,\n",
    "                                           choice_symbols=choice_symbols)\n",
    "    ABuProgress.clear_output()\n",
    "    AbuMetricsBase.show_general(*abu_result_tuple, returns_cmp=only_info, \n",
    "                                only_info=only_info,\n",
    "                                only_show_returns=True)\n",
    "    return abu_result_tuple"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cn_choice_symbols = list(pending_list['Code'])\n",
    "_ = run_loo_back(cn_choice_symbols)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 我吐了 \n",
    "$\\vdots$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "狗股理论使用的是参考值为股息率，很多基于狗股理论的选股策略进行了基因变种，如使用PEG替换股息率进行选股，或者直接使用上一年度的涨幅值做为选股参数，基于基本面数据进行选股的示例将在之后的章节进行示例，本节首先基于涨幅值做为狗股选股的参数。\n",
    "\n",
    "abupy 中内置的选股因子AbuPickStockNTop即是在选股周期上对多只股票涨跌幅进行排序，选取top n 个股票做为交易目标，如下示例使用`AbuPickStockNTop` 在选股周期内选择涨幅最大的top3做为交易目标，如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from abupy import AbuPickStockWorker, AbuPickRegressAngMinMax\n",
    "from abupy import AbuBenchmark, AbuCapital, AbuKLManager, ABuRegUtil,ABuPickStockExecute\n",
    "\n",
    "# 从kl_pd_manger缓存中获取选股走势数据，\n",
    "# 注意get_pick_stock_kl_pd()为选股数据，get_pick_time_kl_pd()为择时\n",
    "kl_pd_noah = kl_pd_manger.get_pick_stock_kl_pd('601988')\n",
    "# 绘制并计算角度\n",
    "deg = ABuRegUtil.calc_regress_deg(kl_pd_noah.close)\n",
    "print('601988 选股周期内角度={}'.format(round(deg, 3)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 选股条件threshold_ang_min=0.0, 即要求股票走势为向上上升趋势\n",
    "stock_pickers = [{'class': AbuPickRegressAngMinMax,\n",
    "                  'threshold_ang_min': 0.0, 'reversed': False}]\n",
    "\n",
    "# 从这几个股票里进行选股，只是为了演示方便\n",
    "# 一般的选股都会是数量比较多的情况比如全市场股票\n",
    "\n",
    "benchmark = AbuBenchmark()\n",
    "capital = AbuCapital(1000000, benchmark)\n",
    "kl_pd_manger = AbuKLManager(benchmark, capital)\n",
    "stock_pick = AbuPickStockWorker(capital, benchmark, kl_pd_manger,\n",
    "                                choice_symbols=choice_symbols,\n",
    "                                stock_pickers=stock_pickers)\n",
    "stock_pick.fit()\n",
    "# 打印最后的选股结果\n",
    "stock_pick.choice_symbols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
